Learning transformed product distributions

We consider the problem of learning an unknown product distribution X over {0, 1}n using samples f (X) where f is a known transformation function. Each choice of a transformation function f specifies a learning problem in this framework. Information-theoretic arguments show that for every transformation function f the corresponding learning problem can be solved to accuracy ǫ, using Õ(n/ǫ2) examples, by a generic algorithm whose running time may be exponential in n. We show that this learning problem can be computationally intractable even for constant ǫ and rather simple transformation functions. Moreover, the above sample complexity bound is nearly optimal for the general problem, as we give a simple explicit linear transformation function f (x) = w · x with integer weights wi ≤ n and prove that the corresponding learning problem requires Ω(n) samples. As our main positive result we give a highly efficient algorithm for learning a sum of independent unknown Bernoulli random variables, corresponding to the transformation function f (x) = ∑n i=1 xi. Our algorithm learns to ǫ-accuracy in poly(n) time, using a surprising poly(1/ǫ) number of samples that is independent of n. We also give an efficient algorithm that uses log n · poly(1/ǫ) samples but has running time that is only poly(log n, 1/ǫ).